Abstract:

Peer review is the backbone of scientific research. It is however faced with a number of challenges which cause unfairness to authors, and degrade the overall quality of the process. This talk will present principled and practical approaches to battle these demons in peer review:

(1) Subjectivity: How to ensure that all papers are judged by the same yardstick?

(2) Mis-calibration: How to use ratings in presence of arbitrary or adversarial mis-calibration?

(3) Bias: How to rigorously test for existence of (gender/fame/race/...) biases in peer review?

(4) Strategic behavior: How to insulate peer review from strategic behavior of author-reviewers?

(5) Noise: How to assign reviewers to papers to ensure fair and accurate evaluations under review noise?